Skip to content

fix(hip): share MatMul and Gemm shapes with dynamic batch validation - #640

Draft
fhanuman wants to merge 1 commit into
pr/shape-stack-01b-reshape-provenancefrom
pr/shape-stack-02-matmul-gemm
Draft

fix(hip): share MatMul and Gemm shapes with dynamic batch validation#640
fhanuman wants to merge 1 commit into
pr/shape-stack-01b-reshape-provenancefrom
pr/shape-stack-02-matmul-gemm

Conversation

@fhanuman

@fhanuman fhanuman commented Aug 3, 2026 •
edited
Loading

Copy link
Copy Markdown
Collaborator

Summary

Unify MatMul, Gemm, and MatMulNBits result-shape construction across ONNX conversion, HIP reification, and static verification.

  • derive MatMul batch/M/N dimensions and transpose-aware Gemm extents from shared semantic helpers;
  • support independent whole-matrix batch broadcasting for A and B;
  • pass operand batch counts and strides through the final hipBLASLt runtime ABI;
  • reject dynamically concealed partial-batch layouts through the recoverable runtime error path;
  • keep converter, dialect, lowering, real/mock runtime, and ABI changes atomic.

Compiled model artifacts using the previous MatMul wrapper ABI must be invalidated.

Stack

This is PR 4 of 12 and depends on PR 3:

1. Constant carrier/externalization
2. Shape-rule foundation
3. Reshape provenance
4. MatMul/Gemm ← this PR
5. Broadcast/Reduction
6. Loop carriers
7. Payload/Readback
8. Conv/Pool
9. Gather/Tensor
10. Attention/Normalization
11. Explicit contract enforcement
12. Refinement hardening

AI assistance

AI tools assisted with the refactoring and stack reconstruction. The complete compiler/runtime ABI slice was checked with the authoritative local build and focused MatMul/Gemm tests.

Made with Cursor

@github-actions

github-actions Bot commented Aug 3, 2026

Copy link
Copy Markdown

Thanks for opening a PR!

This project follows LLVM's incremental-development and AI-tool-use
guidance. See CONTRIBUTING.md
for the project workflow.

Before requesting review, please check that:

  1. The change is focused. Substantial work links the relevant issue
    or design discussion.
  2. The PR documents relevant test results and updates affected
    documentation.
  3. If AI tools provided substantial assistance, the description
    explains what was assisted and how it was validated, and commit
    trailers identify the tool. The contributor has reviewed and
    understands the result.

Reviewers are assigned through
CODEOWNERS where ownership
is configured.

@github-actions github-actions Bot added the large-pr Soft size threshold (>500 LOC or >10 files). Reviewer signal, not a block. label Aug 3, 2026
@github-actions

github-actions Bot commented Aug 3, 2026 •
edited
Loading

Copy link
Copy Markdown

L2 Accuracy Results (EP vs CPU)

Model Combined L2 Total Elems Skipped NaN/Inf
conv_test_hybrid 4.8668E-07 64 0
GroupQueryAttention_seq256 25.2366 2621440 0
MatMulNBits_o_seq128 259.906 368640 0
QMoE_seq128 34.9551 368640 0

Threshold: 0.01 | Run: 3963 - Commit: 210258f

@github-actions

github-actions Bot commented Aug 3, 2026 •
edited
Loading

Copy link
Copy Markdown

MorphiZen EP Performance Results

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.52 6.29 365 3 1244
GroupQueryAttention_seq128 4549.22 1.73535 12 6 313
matmul_down_seq128 499.68 2.38 72 3 354

EPContext Export Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.57 44.19 359 3 15589

EPContext Import Performance

Model QPS Session (s) 1st Infer (ms) CPU% Mem (MB)
full_model_seq128 7.53 9.19 358 3 15759

OGA Benchmark Results

Model Warmup Reps Prompt Len Gen Tokens TTFT (ms) TPS Peak Mem (GB) GPU Mem (GB)
gpt-oss-20b-webgpu-int4-rtn-block-32 1 5 128 128 162.6 80.6 1.33 13.53
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 1 5 128 128 339.3 40.3 1.22 6.43

OGA Wheel Smoke (Python benchmark_e2e.py)

Model TTFT (ms) TPS
Llama-3.1-8B-awq-g128-int4-asym-fp16-onnx-dml 127 39.6

Run: 3963 - Commit: 210258f

@fhanuman
fhanuman force-pushed the pr/shape-stack-02-matmul-gemm branch from 9312980 to f4f5cc3 Compare August 5, 2026 21:56
@fhanuman
fhanuman force-pushed the pr/shape-stack-02-matmul-gemm branch 2 times, most recently from a7dd44e to 6cbba55 Compare August 7, 2026 19:51
@fhanuman
fhanuman force-pushed the pr/shape-stack-02-matmul-gemm branch from 6cbba55 to c217451 Compare August 7, 2026 20:25
@fhanuman
fhanuman changed the base branch from pr/shape-stack-01-foundation to pr/shape-stack-01b-reshape-provenance August 7, 2026 22:12
@fhanuman
fhanuman force-pushed the pr/shape-stack-02-matmul-gemm branch 2 times, most recently from ea4515c to f519c86 Compare August 10, 2026 22:29
@fhanuman
fhanuman force-pushed the pr/shape-stack-02-matmul-gemm branch 2 times, most recently from 52f86a3 to 9d7fdec Compare August 11, 2026 00:42
@fhanuman @codex
fix(hip): share MatMul and Gemm shapes with dynamic validation
210258f 
Unify converter, reification, verification, lowering, and runtime behavior while validating dynamic contraction extents and every broadcast batch axis before checked dispatch.

Co-Authored-By: GPT-5.6 Sol <noreply@openai.com>
Made-with: Cursor
@fhanuman
fhanuman force-pushed the pr/shape-stack-02-matmul-gemm branch from 9d7fdec to 210258f Compare August 11, 2026 16:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@amd-mingw amd-mingw Awaiting requested review from amd-mingw amd-mingw will be requested when the pull request is marked ready for review amd-mingw is a code owner
@edelaye edelaye Awaiting requested review from edelaye edelaye will be requested when the pull request is marked ready for review edelaye is a code owner
@wcy123 wcy123 Awaiting requested review from wcy123 wcy123 will be requested when the pull request is marked ready for review wcy123 is a code owner
@qianglin-amd qianglin-amd Awaiting requested review from qianglin-amd qianglin-amd will be requested when the pull request is marked ready for review qianglin-amd is a code owner
@amd-bqiao amd-bqiao Awaiting requested review from amd-bqiao amd-bqiao will be requested when the pull request is marked ready for review amd-bqiao is a code owner

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

large-pr Soft size threshold (>500 LOC or >10 files). Reviewer signal, not a block.

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@fhanuman